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Abstract 

A spectral method for identifying lumping in large Markov chains 
is presented. Identification of meta stable states is treated as a special 
case. The method is based on spectral analysis of a self-adjoint matrix 
that is a function of the original transition matrix. It is demonstrated 
that the technique is more robust than existing methods when applied 
to noisy non-reversible Markov chains. 

1 Introduction 

The structural dynamics of large biomolecules can often be accurately de- 
scribed as a Markov transition process. Frequently, the dynamics display 
separation of time scales where aggregated conformational states are evolv- 
ing at much slower rate than the detailed molecular dynamics does. The 
problem of identifying the conformational states from the detailed Markov 
transition matrix has received recent interest [TJ [2j EJ Hj. The technically 
similar problem of identifying modularity and community structure on com- 
plex networks has also been discussed extensively, e.g. 0E1E]. 

Identification of meta stable states is a special case of a more general 
reduction called (approximate) lumping. Lumping of a Markov chain means 
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that the state space is partitioned into equivalence classes of states called 
macro states. A coarse grained process is defined by the transitions between 
the macro states. If the coarse grained process is Markovian, i.e. exhibits no 
memory, we call the reduction a lumping. A partition into meta stable states 
is an example of an approximate lumping in the following sense. In the limit 
of complete stability, i.e. when there are no transitions between the macro 
states, then the macro states define a degenerate case of exact lumping. 
More generally, the Markov property is fulfilled on the aggregated level if 
the relaxation process within a meta stable state is fast and mixing so that 
the memory of exactly how the meta stable state was entered is lost before 
the transition to a new meta stable state occurs. In this sense aggregation 
into meta stable states can be viewed as an approximate lumping. Aside 
from separation of time scales, there are other generic situations when a 
Markov process is expected to be lumpable. For example when a particle 
interacts with many other particles, a "heat bath" , the dynamics of the single 
particle can be described as a Brownian motion. Technically the transition 
matrix of a lumpable Markov chain can be rearranged into a block-stochastic 
structure, see Fig. [5] and definition (]13p . Markov chains with metastable 
states can be permuted into a block-diagonal structure (Fig. [1]), which is a 
special case of a block-stochastic matrix. 

The most successful methods for identifying meta stable states and mod- 
ules in networks are based on the level structure of the eigenvectors whose 
corresponding eigenvalues are clustered close to the Perron-Frobenius eigen- 
value. The technique introduced in this paper is closely related to these 
spectral method first introduced by Fidler in the 70's, at that point as a 
method for graph partitioning [5]. Fidler noted that the second eigenvector 
of the graph Laplacian shows tightly connected communities of nodes that 
are connected to the other communities by relatively few edges, or low alge- 
braic connectivity. Later the method was used in connection to the classic 
graph coloring problem [8]. In the same paper the idea of using the sign 
structure of the k first eigenvectors to partition a graph into k aggregates 
of nodes was introduced. The same idea was later applied to identify meta 
stable states in Markov chains [T]. For these spectral methods to be stable, 
the eigenvalue problem must be symmetric with respect to some scalar prod- 
uct. This means that the Markov process must be reversible, or that the 
network is assumed to be effectively non-directional. A notable exception in 
presented based symmetrization using the stationary distribution was pre- 
sented in [9] . Another exception is a recent method for Markov chains based 
on singular value decomposition of the Markov transition matrix [3]. How- 
ever, the SVD-based method is not appropriate for identifying lumpings of 
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Markov chains since the singular vectors typically do not have a level struc- 
ture (or relevant sign structure) in the theoretical limit of exact lumpability, 
see for example the transition matrix defined in ([2]). 

In this paper we present a new robust spectral method for identifying 
possible lumpings of non-reversible Markov chains. Instead of using the 
spectrum of the transition matrix directly, we define a self-adjoint "invari- 
ance matrix" whose kernel relates to the eigenvectors that define the meta 
stable states, or more generally the lumps of the Markov chain. Since the 
invariance matrix is self-adjoint by construction, the usual assumption of re- 
versibility can be lifted. We demonstrate the method of both Markov chains 
with meta stable states and more general block-stochastic structure, and 
compare the performance to other methods reported in the literature, e.g. 
the methods presented in [9] and [4]. 



2 Lumping of Markov chains 

Consider a Markov process xt+i = XfP. The N x N transition matrix P 
is a row stochastic matrix, i.e. Y,j Pij = 1 Vi. A lumping is defined as a 
partition of the states space £ into K equivalence classes of states such 
that -Lfc n L\ = and u^Lfc = S |10| . A necessary and sufficient condition for 
a partition to be a lumping is [11] 

£ Pij constant for all i in an aggregate, i e L^. (1) 
j^U 

If a Markov chain allows for a non-trivial lumping we call it lumpable. A 
simple example of a lumpable transition matrix is 




which, aside from the trivial lumping defined by all states aggregated into 
one macro-state, allows the non-trivial lumping {{1,2}, 3}, i.e. state 1 and 
2 lumped into one macro-state. 

The condition in ([TJ also immediately defines the transition matrix for 
the aggregated dynamics 

Phi = £ Pij i£L k , (3) 
j^U 
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since all states i e L,t give the same result. In practice Eq. Q]is usually not 
fulfilled exactly. For example, if a transition matrix can be written as 

P = (l-e)A + e£, (4) 

where A is a transition matrix that fulfills the lumpability condition ([T]) and 
B is some arbitrary transition matrix. Then, if e is small, we say that P is 
approximately lumpable. Note that the aggregated transition probabilities 
in ([1]) are in this case approximately constant with deviations of 0(e). The 
reduced dynamics must in this case be approximated e.g. using a weighted 
average for the transitions between the aggregated states 

where Vj is the stationary distribution. Using the weighted average is natural 
since it gives the same reduced transition matrix as we find if we estimate 
the aggregated transition probabilities directly from a stationary time series. 

A partition can be represented by a matrix II defined as 11^ = 1 if i e Lf. 
and Iljjfc = otherwise. Eq. (pQ) can be reformulated as 

PIT = HP, (6) 

which, if written out explicitly in terms of the elements, implies that the 
column space of II spans a right-invariant subspace of P. Assuming that 
P is diagonalizable, the invariant subspace is spanned by a set of right 
eigenvectors of P, and due to the or 1 structure of II the elements in 
these eigenvectors must be constant over the aggregates. To be more pre- 
cise, a lumping with K aggregates exists if and only if there are exactly K 
right eigenvectors of P with elements that are constant over the aggregates, 
see [X2(. fl~3] for details. As an example, the transition matrix defined in (j2j) 
allows for the lumping {{1,2}, 3} as indicated by the two first elements in 
the right eigenvectors (1,1, 1) T and (-1,-1, 2) T being constant. 

It should be noted that there exist other types of aggregation of states 
where the aim is to preserve (for example) the structure of the equilib- 
rium distribution. A prominent example of this is renormalization of lattice 
spin systems. However, in this paper we focus on lumping that respect the 
dynamics of the process. In this case the Markov property is the central con- 
straint, i.e. the mutual information between the past and the future given 
the present should be zero on both the micro (a prerequisite for the pro- 
cedure) and macro level (the lumping condition). This leads to the strong 
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conditions on the aggregation seen in Eq. Q] and Eq. [6j For a more detailed 
discussion on how memory appears on the coarse grained level if the lumping 
criterion is not fulfilled, see [TB] . 

The principle idea behind spectral methods for identifying lumping or 
meta stable states, as well as modularity in networks, is to search for (right) 
eigenvectors whose elements are constant over the aggregates, i.e. eigen- 
vectors with a level structure, see e.g. |13| for details. If the transition 
matrix is symmetric under some scalar product the eigenvectors are orthog- 
onal and it is easy to show that the constant level sets must have different 
sign structure [8J (the sign structure of a vector is defined by mapping neg- 
ative elements to -1 and positive elements to +1). The sign structure is 
often used as a lumping criterion rather than the constant levels since this 
is expected to be a more numerically stable [8j Hj. However, a more recent 
study has shown that the sign structure is more sensitive to noise than the 
constant level structure over the aggregates, an observation that lead the 
authors to introduce an algorithm based on the simplex structure of the 
almost constant level sets [2]. 

For the detection of metastable states or modularity, the eigenvectors of 
interest are those corresponding to eigenvalues close to the Perron- Frobenius 
eigenvalue, since these eigenvalues are related to the slow dynamics. In 
the case of general lumping the eigenvectors involved are not necessarily 
distinguished by their appearance in the spectrum. However, as we discuss in 
Section [5j for large transition matrices the eigenvectors involved in lumping 
tend to be separated from the rest of the spectrum by being located further 
from the origin in the complex plane than the rest of the spectrum, but not 
necessarily by being closer to 1. 

As a complement to the spectral methods, the commutation relation © 
can be used directly to identify lumping of Markov chains. Start by making 
a random assignment of the N states to K aggregates, and construct the 
corresponding n matrix. Given the n matrix, the reduced transition matrix 
P, defined in Eq. [3] can be derived by simply ignoring that the row elements 
are not constant within the aggregates and use the average defined in ([5]). 
The left hand side in ([6]), PT1, defines a K dimensional vector for each of the 
N states. If the lumping is correct then all states in an aggregate k should 
have identical K dimensional vectors, and they should be equal to the fcth 
row of P. If n is not a lumping we can try to improve it by assigning state i to 
aggregate k where k = argmin;||(Pn)j -P||. The result is a new aggregation 
with a new n matrix. The process can be iterated until convergence. A 
similar method was introduced by Lafon and Lee [13] • As pointed out in 
[15j it is similar in structure to the if -means clustering algorithm. It should 
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be noted that this direct clustering technique only works if the aggregated 
dynamics has long relaxation time, i.e. there is a spectral gap supporting 
the lumping, see [13] for details. The performance of the algorithm is shown 
in comparison with the method introduced in this paper in Fig. [3] and [7J 

3 A robust method for identifying lumping 

We now present the main idea of this paper. We would like to find invariant 
vectors containing invariant level sets. For moderately sized (unperturbed) 
transition matrices or for time-reversible Markov chains, the eigenvectors 
can be used to detect lumping. If the Markov chain is not reversible, calcu- 
lation of both eigenvalues and eigenvectors is numerically unstable, since, for 
example, the transition matrix may contain non-trivial Jordan blocks [13] . 
This is the motivation for the new method. Start by noting that, if a nor- 
malized vector u is approximately right invariant under P, then there must 
exist a A such that 

\\(P-XI)u\\ 2 «l, (7) 

where / denotes the identity matrix. The square of the 2-norm on the 
left hand side in ([TJ is not sensitive to small changes in the elements of 
P, whereas if P is non-symmetric the eigenvalues and eigenvectors can be 
ill conditioned. Obviously, if u is an eigenvector and A the corresponding 
eigenvalue, then ([7]) is zero, reflecting the fact that the eigenvector is exactly 
invariant. The 2-norm of ([7]) is given by 

n f Q(AK (8) 

where u' denotes the conjugated transpose of the vector u. The "invariance 
matrix" Q is defined as 

Q(X) = P^P - A*P - AP t + |A| 2 /, (9) 

(note that Q is typically not a stochastic matrix) . Regardless of the proper- 
ties of P, Q(X) is by construction a self-adjoint matrix and diagonalization 
is numerically stable. If A is an eigenvalue of P, then Q(A) is positive semi- 
definite with a zero eigenvalue corresponding to the eigenvector of P with 
eigenvalue A. If A is not an eigenvalue, then Q(X) is positive definite. For a 
given A, ([7]) is minimized by u being the eigenvector of Q(X) corresponding 
to the smallest eigenvalue of Q(X), or in the case of degeneracy a linear 
combination of the eigenvectors of the smallest eigenvalue. 
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4 Meta stable states 



Detecting meta stable states is an especially simple, but also especially in- 
teresting, case of general (approximate) lumping. The meta stable states 
are characterized by their long relaxation time, and hence their dynamics 
is associated with eigenvalues close to 1. The right eigenvectors involved in 
the aggregation have corresponding eigenvalues closer to 1 than the rest of 
the spectrum. As a consequence, meta stable states can be identified by the 
approximate constant level structure of the eigenvectors of 



with eigenvalues close to 0. As a consequence, the small eigenvalues of (|10p 
include the eigenvectors needed in the aggregation. It should be noted that 
the actual eigenvalues of P associated with the meta stable states need not 
be close to 1 for the sub-dominant eigenvectors of Q(l) to reveal the meta 
stable states, see Fig. [2]and[3l Since Q is self-adjoint the eigenvectors are or- 
thogonal. The eigenvectors are approximately constant over the aggregates, 
and orthogonality can then only be achieved if each aggregate has a unique 
sign structure in the eigenvectors. This observation was used by Aspvall 
and Gilbert [8] and Deuflhard and coworkers [H[2], but in these cases under 
the condition of symmetry of the adjacency matrix or reversibility of the 
transition matrix respectively. Using the Q matrix there is no need to make 
assumptions on P. It is straight forward to apply the same sign structure 
criterion to the eigenvectors of Q, but empirical tests have shown that in 
our case the following simple approach is relatively robust (see Fig. H|) : 

1. Find the eigenvectors {ui}f =l corresponding to the small eigenvalues 
of Q(l) in CEO]). 

2. For each state j = 1 , . . . , N form a K dimensional vector u,j = (u%j , U2j , . . . , UKj ) 
of its corresponding elements in the K eigenvectors. 

3. Use a standard clustering algorithm (e.g. K-mean) to cluster the 
states with respects to the u,j vectors. Note that we expect the level 
structure in the eigenvectors to be relatively stable to perturbations 
(0(e 2 )), as pointed out in [2]. 

To test the method we generate two classes of matrices. The first class 
is on the form 



Q(l) = P ] P-P-P ] + 1 



(10) 



P = (l-e)B + eA, 
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where B is a block diagonal transition matrix with 3-5 blocks and transition 
probabilities within the blocks chosen uniformly in the interval [0, 1] and 
then normalized. The matrix A is a transition matrix with no block diagonal 
structure, generated in the same way as the blocks in B. The parameter 
e sets the level of perturbation of P from being block diagonal. Fig. [T]- 
[3] show an example of a transition matrix of this type with e = 0.7 and 
the corresponding spectrum and clustering of the elements in the dominant 
eigenvectors of P and the sub-dominant eigenvectors of Q. 

It can be argued that matrices of the type in (fTTI) are unlikely to appear 
in practical applications. Instead of the smooth average modulation that 
the decrease transition probabilities between blocks in (jllj) . a more binary 
modulation often occurs in practice, i.e. many transition probabilities are 
zero. In this situation meta stable states occur as a consequence of a higher 
probability of having transitions within (rather than between) meta stable 
states. Contrasting the construction in (|lip this produces a sparse transition 
matrix. We construct the second class of matrices according to 

P*j(e,S)= X ij(e,S)B ij , (12) 

where B, as before, is a matrix with random entries chosen uniformly in 
the interval [0,1]. In the matrix x(e,<5), the entries are binary chosen so 
that Xij - 1 with probability S if i and j are in the same block, and Xij - 1 
with probability e if i and j are not in the same block, otherwise Xij - 0- 
Thus 5 controls the overall probability of transitions within the blocks and 
e controls the transitions between the blocks, with S > e. The two extreme 
points are e = which produce a completely block diagonal matrix, and e = 5 
which gives a matrix without any block diagonal structure. The rows in the 
matrix P* is normalized to produce a stochastic matrix. The procedure 
described can produce states with no outgoing transitions, i.e. that are ill 
defined as transition matrices. If this happens we generate a new matrix. 

We tested the performance of the Q method and compared it to the fol- 
lowing existing techniques: results from the eigenvectors of P, the right and 
left singular vectors from an S VD as suggested in [1] , the clustering method 
presented in |14| . and the spectral method described in [9j. As test cases we 
used the two classes of matrices described above and measured how stable 
the meta stable states produced by the different methods where, i.e. the 
average waiting time between jumps between meta stable states. The result 
is shown in Fig. HJ For each value of e shown, the average switching time 
is measured for 100 matrices of size 200 x 200 in respective class. The time 
r is scaled so that the "correct partitioning" used to generate the matrices 
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Figure 1: A weakly block dominant transition matrix, constructed as (jlip 
with e = 0.7, is shown. The time scale separation is not very pronounced, 
see the spectrum in Fig. [3j To the left with random permutation and to the 
right after sorting the matrix according to the aggregation revealed in the 
clusters of the eigenvectors of the Q matrix shown in Fig. [2j 



have waiting time 1. The results indicate that the Q method is more robust 
against perturbations than previously reported methods. It is especially in- 
teresting to note that for high e values, i.e. when the original block diagonal 
structure is almost lost, the Q method still produce aggregations that are 
more stable than random partitions. Non of the other methods are capable 
of finding these very weak meta stable states. 



5 Block stochastic matrix 



As discussed earlier, matrices with dominant block diagonal structure are 
special cases of lumpable Markov processes. The more general structure of 
lumpable transition matrices is shown in Fig. \E\ A block-stochastic matrix 
is a matrix on the form 



P = 



' -PllCtll P\2&\2 ■•■ PlmO-lm 

Pmm,0"mm ) 



(13) 



where P is the mx m transition matrix of the reduced dynamics and each 
of the Q>ij IS cl transition matrix in itself. Naturally, ajj and ciji must for a 
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Figure 2: The figure shows the clustering of the elements in the second and 
third smallest, respective largest, eigenvectors of Q(l) (to the left) respec- 
tively P (to the right) of the matrix shown in Fig. [TJ Note that the clusters 
are more distinct in the eigenvectors of Q(l) shown on the left. 




Figure 3: The spectrum of the transition matrix P in Fig. Q] to the left 
and of the corresponding Q(l) matrix on the right. The aggregation into 
meta stable states is associated with the dominant eigenvalues of P, i.e. 
the Perron-Frobenius eigenvalue and the two eigenvalues close to 0.1, or 
alternatively with the three smallest eigenvalues of Q(l) (to the far right in 
the figure). Note that even though the dominant eigenvalues of P are not 
clustered close to 1, the spectrum and eigenvectors of Q(l) show the meta 
stable states more clearly than the eigenvectors of P, see Fig. [2j 
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Figure 4: The average result of identifying the meta stable states of a domi- 
nant block diagonal transition matrices defined in (jlip is shown on the left, 
and for matrices defined in (|12|) with 5 = 0.2 on the right. The average 
waiting time for transitions between meta stable states (normalized against 
the result for the partitioning used to generate the matrices) for 100 test 
matrices of size 200 x 200 for each e value is used as a measure of the quality 
of the results. The result for the Q method is displayed as •, the result for 
using eigenvectors of P as ■. the singular vectors from SVD [1] as ♦, results 
from the clustering method suggested in [H] (see Sec. [2] for details) as ▼, 
and the method suggested in [9] as A. 



11 



fixed i have the same dimensions for j = 1, . . . , m. 

The spectra of large block stochastic matrices tend to separate out the 
eigenvalues associated with the lumping. The separation is however different 
from the one occurring in block diagonally dominant transition matrices. 
Instead of clustering around the Perron-Frobenius eigenvalue, the reducing 
eigenvalues of a block stochastic matrix separate by larger distance to the 
origin in the complex plane, see Fig. [6j The reason behind the separation is 
also different from the block diagonal case, and only appears as a statistic 
effect for large random transition matrices, as the following argument shows. 
When lumping a Markov chain, the spectrum of the reduced dynamics is 
always a subset of the original spectrum [IB] . For a large transition matrix, 
size JVxJV, with uncorrelated random transition probabilities, the spectrum 
is typically concentrated to a disk with radius ~ l/(2\/~N), except for the 
Perron-Frobenius eigenvalue. For a block stochastic matrix the eigenvalues 
of the lumped Markov chain P are typically concentrated to a disk with 
radius ~ l/(2\/~K) where K is the number of states in the lumped chain. 
If K « N then it should be expected that the eigenvalues associated with 
the lumped process separate from the rest of the spectrum. An example of 
this can be seen in Fig. [6j However, it should be noted that the separation 
is only a typical behavior, it is not necessary for the Markov chain to be 
lumpable (this seems to be incorrectly stated in [T2J Q2], see [13] for details). 

If the spectrum does not show any separated eigenvalues that indicate 
the best choice of A in Q(A) the implementation of the Q method is less 
straight forward when searching for general lumping. This is of course also 
the case for other spectral methods if the eigenvalues involved in the lumping 
does not separate from the rest of the spectrum. Perhaps the easiest way is 
to choose a set of {Aj}^ 1 randomly in the disk of radius 1 in the complex 
plane, use the eigenvectors of Q(Aj), i = 1,...,K, corresponding to the 
smallest eigenvalue, cluster the elements in the same way as for the meta 
stable states, and check how well the result satisfies the lumpability criterion 
(PQ). The procedure must be repeated a few times to find the configuration 
with the most satisfying result. It is possible to design more sophisticated 
methods by re-using the A's that seem to produce good results. However, 
we use the simplest possible approach choosing between 2 and 5 (guided by 
the separation in the spectrum) A values randomly in the complex plane 
and repeating 10 times. The results are shown in Fig. [7] in comparison with 
other methods. In these numerical test we use the deviation from fulfilling 
the lumpability condition © 

A = ||pn-np|| 2 (14) 
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Figure 5: A block stochastic transition matrix P, constructed as (|13p with 
e = 0.5, is shown to the left and PP to the right. The SVD method from [1] 
is based on the right matrix and it is clear that the two matrices share the 
same block stochastic structure. However the numerical tests show that Q 
method based on the left matrix is more stable to perturbations than the 
SVD method based on the eigenvectors of the right matrix. The spectrum 
of P is shown in Fig. [H 

Im(A) 



; Re(A) 



-0.4 -0.2 0.0 0.2 0.4 0.6 0.8 



Figure 6: The figure shows the spectrum of the block stochastic matrices 
shown in Fig. The eigenvalues associated with the eigenvectors that are 
involved in the lumping process are separated from the rest of the spectrum, 
but typically not close to the Perron-Frobenius eigenvalue. It should be 
noted that though the separation in the spectrum typically appears for large 
block stochastic matrices, this is a statistical effect and not necessary for the 
transition matrix to be lumpable, see the main text for further discussion 
on this. 
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Figure 7: The average result of inferring lumping of states of a block stochas- 
tic transition matrices defined in (|13|) . The deviation from fulfilling the 
lumpability condition, defined in (|14p . is normalized against the deviation 
produced by the lumping used when producing the matrix (i.e. smaller val- 
ues implies better results). For each e value 100 independent realizations of 
200 x 200 matrices on the form (fT3j) was used to calculate the average per- 
formance. The result for the Q method is displayed as •, the result for using 
eigenvectors of P as the singular vectors from SVD [4] as ■. and results 
from the clustering method suggested in [2] (see Sec. [2] for details) as A. 
For moderate e the Q method and the clustering performs equally superior 
to the other methods, while for larger e all methods except the clustering 
technique show approximately equal performance. 
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as a measure of how well the different methods perform. For each value 
of e test where performed with 100 matrices generated on the form P = 
(1 - e)B + eA, where B was constructed according to (|13p and B was a 
random transition matrix. The numerical tests indicate that the Q method 
is more stable than using P directly or the SVD method. The clustering 
method performs almost as well as the Q method. 

From a computational perspective the Q method is, in the case of general 
lumping, significantly slower than the other spectral methods. The reason 
is that several random choices of A's must be tried and in addition Q(X) 
is a complex matrix if A is complex. Neither of these complications occur 
when searching for meta stable states since then we know beforehand that 
A = 1 is a good choice. In the implementation used to produce the results in 
Fig. [7] the Q method is approximately 15 times slower than using P directly 
or the SVD method. On the other hand the results are also better. The 
slowdown scales proportional to the number of A-setups we need to try. 
A more sophisticated selection procedure for choosing the regions where 
the sub-dominant eigenvectors of Q(A) show a clear signal would probably 
increase the efficiency of the algorithm. 

6 Conclusions 

We have introduced a new spectral method for identifying lumping in large 
Markov chains, with the identification of meta stable states as an important 
special case. The key element of the method is to define a family of self- 
adjoint matrices from the transition matrix. The eigenvectors of the self- 
adjoint matrices are, as opposed to those of the transition matrix itself, 
stable to perturbations or noisy estimation of the transition probabilities. 
The robustness of the method is tested and compared to the results from 
previous methods, including a direct clustering method introduced in [14] 
and the recently suggested SVD based method introduced in [3]. The Q 
method is shown to be more robust than previous techniques. 

We mentioned in the introduction that the method presented here can 
be used to reduce networks by aggregating nodes. The examples in this 
paper are however focused completely on lumping of Markov chains. The 
relation between lumping of Markov chain and reduction of complex net- 
works was recently discussed in |15j . The authors define a diffusion process 
on the network using the standard method of the graph Laplacian. It should 
be noted however that reduction of networks can be defined with respect to 
other types of dynamics than diffusion. Straight forward jump processes 
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are, for example, defined directly by multiplication of the adjacency matrix. 
Since the lumpability condition considered in this paper applies to general 
linear processes, not only Markov processes with stochastic transition ma- 
trix, the methods introduced can be used to reduce networks by aggregation 
with respect to different criteria depending on which dynamic process we are 
considering on the network. For the Q method to be different from using the 
eigenvectors of the transition matrix directly, the graph must be directed. 

References 

[1] P. Deuflhard, W. Huisinga, A. Fischer, and Ch. Schiitte. Identification 
of almost invariant aggregates in reversible nearly uncoupled Markov 
chains. Linear Algebra and its Applications, 315(1-3) :39-59, 2000. 

[2] P. Deuflhard and M. Weber. Robust perron cluster analysis in confor- 
mational dynamics. Linear Algebra and its Applications, 398(15):161- 
184, 2005. 

[3] C.H. Jensen, D. Nerukh, and R.C. Glen. Sensitivity of peptide con- 
formational dynamics on clustering of a classical molecular dynamics 
trajectory. Journal of Chemical Physics, 128:115107, 2008. 

[4] D. Fritzsche, V. Mehrmann, B. Szyld, and E. Virnik. An SVD ap- 
proach to identifying metastable states of Markov chains. Electronic 
transactions on numerical analysis, 29:46-69, 2008. 

[5] M. Fiedler. Algebraic connectivity of graphs. Czechoslovak Mathemat- 
ical Journal, 23:298-305, 1973. 

[6] A. Pothen, H. Simon, and K-P. Liou. Partitioning sparse matrices 
with eigenvectors of graphs. SI AM journal on matrix analysis and its 
applications, ll(3):430-452, 1990. 

[7] M. Newman. Modularity and community structure in networks. Pro- 
ceedings of the national academy of science, 103(23):8577-8582, 2006. 

[8] B. Aspvall and J.R. Gilbert. Graph coloring using eigenvalue decompo- 
sition. SI AM journal on algebraic and discrete methods, 5(4):526-538, 
1984. 

[9] G. Froyland. Statistically optimal almost-invariant sets. Physica D, 
200(3-4) :205-219, 2005. 



16 



[10] L. C. G. Rogers and J. W. Pitman. Markov functions. Annals of 
Probability, 9:573-582, 1981. 

[11] J. G. Kemeny and J. L. Snell. Finite Markov Chains. Springer, New 
York, NY, USA, 2nd edition, 1976. 

[12] J. Shi. A random walks view of spectral segmentation. In In AI and 
Statistics (AISTATS, 2001. 

[13] M. Nilsson Jacobi and O. Gornerup. A spectral method for aggregat- 
ing variables in linear dynamical systems with application to cellular 
automata renormalization. Advances in Complex Systems, 12(2): 1-25, 
2009. 

[14] S. Lafon and A.B. Lee. Diffusion maps and coarse-graining: A unified 
framework for dimensionality reduction, graph partitioning, and data 
set parameterization. IEEE Transactions on Pattern Analysis and Ma- 
chine Intelligence, 28(9): 1393-1403, 2006. 

[15] W. E, T. Li, and E. Vanden-Eijnden. Optimal partition and effective 
dynamics of complex networks. Proceedings of the National Academy 
of Sciences, 105(23), 2008. 

[16] D Barr and M Thomas. An eigenvector condition for markov chain 
lumpability. Operations Research, 25(6): 1028-1031, 1977. 



17 



